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What is claimed is: 



1 1. A method for dynamically developing a marketing strategy to address at least one 

2 specified merchant objective, the objective corresponding to a specified time period and a 

3 specified budget, the strategy being implemented across at least one marketing channel, 

4 the strategy including at least one initiative, the method comprising the steps of: 

5 a. generating a plurality of marketing strategies; 

6 b. determining an optimal marketing strategy based on state of a customer and 

7 constraints corresponding to marketing channels; 

8 c. deploying the determined optimal marketing strategy; 

9 d, recording customer response to the deployed marketing strategy; 

10 e. updating information corresponding to the state of a customer based on the 

1 1 recorded customer response; and 

12 f. repeating steps b to e for the specified time period. 

1 2. The method as recited in claim 1 wherein the step of generating marketing strategies 

2 comprises the steps of: 

3 a. selecting at least one initiative that enables the addressing of the specified 

4 objective; 

5 b, determining the sequences in which the selected initiatives can be deployed, if 

6 more than one initiative is selected; and 

7 c. combining the initiatives in the determined sequences to obtain the plurality of 

8 marketing strategies. 

1 3. The method as recited in claim 2 further comprising the step of varying the parameters of 

2 initiatives to generate new initiatives. 

1 4. The method as recited in claim 2 further comprising the step of varying deployment time 

2 of initiatives. 

1 5. The method as recited in claim 1 wherein the step of determining an optimal marketing 

2 strategy further comprises the steps of: 

3 a. determining all possible states of customers; 
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4 b, determining an optimal policy for each state based on past data; 

5 c. identifying the state of a customer, the customer visiting the merchant or the 

6 customer being selected from a database of customers; and 

7 d. identifying an optimal marketing strategy using the state of the customer, the 

8 identified optimal policy and constraints corresponding to marketing channels. 

1 6. The method as recited in claim 5 wherein the step of identifying all possible states of 

2 customers comprises the steps of: 

3 a. identifying all relevant attributes of customers; and 

4 b. partitioning the customers based on the identified attributes using a similarity 

5 measure based on a historic policy, actual rewards and transition probabilities 

6 from one data point to another, the partitions forming the new states of the 

7 customers. 

1 7. The method as recited in claim 5 wherein the step of determining the optimal policy for 

2 each state based on past data comprises the steps of: 

3 a. identifying a deterministic policy; 

4 b. initializing the value of all possible states for the policy; 

5 c. computing the value of a state for the policy; 

6 d. repeating step c for all possible states; 

7 e. constructing a new improved policy; 

8 f. iteratively performing steps c to e until the new improved policy remains 

9 unchanged for two subsequent iterations; and 

10 g. selecting the policy with maximum value for the state as the optimal policy for the 

1 1 given state. 

1 8. The method as recited in claim 7 wherein the step of computing the value of a state for the 

2 policy comprises the steps of: 

3 a. computing transition probabilities from the given state to another state for the 

4 policy; 

5 b. computing value of expected innmiediate reward for the policy in the state; 

6 c. computing discounted expected value of the resulting state for the policy; and 
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7 d. computing the sum of expected immediate reward and the discounted expected 

8 value. 

1 9. The method as recited in claim 7 wherein the step of constructing a new improved policy 

2 comprises the steps of: 

3 a. selecting the marketing strategy which maximizes the value for the state over all 

4 marketing strategies for a given state; and 

5 b. repeating step a for each state. 

1 10. The method as recited in claim 5 wherein the step of identifying an optimal marketing 

2 strategy comprises the steps of: 

3 a. identifying the optimal policy for the identified customer state; 

4 b. modeling customer's preferences for marketing channels, cost and effectiveness of 

5 different marketing channels, and the specified budget as effective constraints; 

6 c. determining an optimal feasible policy based on the identified optimal policy and 

7 the effective constraints corresponding to marketing channels; and 

8 d. determining the optimal marketing strategy from the optimal feasible policy. 

1 11. The method as recited in claim 10 wherein the step of determining an optimal feasible 

2 policy based on effective constraints corresponding to marketing channels comprises the 

3 step of mapping the optimal policy uniquely to a closest feasible optimal policy based on 

4 the effective constraints, if the constraints are not satisfied by the optimal policy. 

1 12. The method as recited in claim 1 wherein the step of updating information corresponding 

2 to the state of a customer based on the recorded customer response comprises the steps of: 

3 a. identifying the resulting state of the customer; 

4 b, updating the values of the states of the customer; and 

5 c. updating the optimal policy. 

1 13. The method as recited in claim 12 wherein the step of updating the values of the states of 

2 the customer comprises the steps of: 
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3 a. computing the sum of the new immediate reward, the discounted value 

4 corresponding to the resulting state, reduced by the value corresponding to the 

5 initial state of the customer; 

6 b. updating the value corresponding to the initial state of the customer by adding a 

7 fraction of the computed sum to value of previous state of customer; and 

8 c. propagating the change in the value of the state to all other states. 

1 14. The method as recited in claim 12 wherein the step of updating the optimal policy 

2 comprises the steps of: 

3 a. computing the sum of the new inmiediate reward, the discounted value 

4 corresponding to the resulting state, reduced by the value corresponding to the 

5 initial state of the customer; and 

6 b. updating the optimal policy corresponding to the initial state of the customer by 

7 adding a fraction of the computed sum to the value of previous state of customer. 

1 15. A system for dynamically developing a marketing strategy to address at least one 

2 specified merchant objective, the objective corresponding to a specified time period and a 

3 specified budget, the strategy being implemented across at least one marketing channel, 

4 the strategy including at least one initiative, the system comprising: 

5 a. means for generating a plurality of marketing strategies; 

6 b. means for determining an optimal marketing strategy based on state of a customer 

7 and constraints corresponding to marketing channels; 

8 c. means for deploying the determined optimal marketing strategy; 

9 d. means for recording customer response to the deployed marketing strategy; and 

10 e. means for updating information corresponding to the state of a customer based on 

1 1 the recorded customer response. 

1 16. The system as recited in claim 15 wherein the means for generating marketing strategies 

2 comprises: 

3 a. means for selecting at least one initiative that enables the addressing of the 

4 specified objective; 

5 b. means for determining the sequences in which the selected initiatives can be 

6 deployed, if more than one initiative is selected; and 
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7 c. means for combining the initiatives in the determined sequences to obtain the 

8 plurality of marketing strategies. 

1 17. The system as recited in claim 15 wherein the means for determining an optimal 

2 marketing strategy comprises: 

3 a. means for determining all possible states of customers; 

4 b. means for determining an optimal policy for each state based on past data; 

5 c. means for identifying the state of a customer, the customer visiting the merchant 

6 or the customer being selected from a database of customers; 

7 d. means for identifying the optimal policy for the identified customer state; 

8 e. means for modeling customer's preferences for marketing channels, cost and 

9 effectiveness of different marketing channels, and the specified budget as effective 

10 constraints; 

11 f. means for determining an optimal feasible policy based on effective constraints 

12 corresponding to marketing channels; and 

13 g. means for determining the optimal marketing strategy from the optimal feasible 

14 policy. 

1 18. The system as recited in claim 17 wherein the means for determining the optimal policy 

2 for each state based on past data comprises: 

3 a. means for identifying a deterministic policy; 

4 b. means for initializing the value of all possible states for the policy; 

5 c. means for computing the value of a state for the policy; 

6 d. means for constructing a new improved policy; 

7 e. means for iteratively executing means for computing and means for constructing; 

8 and 

9 f. means for selecting the policy with maximum value for the state as the optimal 
10 policy for the given state. 

1 19. The system as recited in claim 18 wherein the means for constructing a new improved 

2 policy comprises means for selecting the marketing strategy that maximizes the value for 

3 the state over all marketing strategies for a given state. 
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1 20. The system as recited in claim 15 wherein the means for updating information 

2 corresponding to the state of a customer based on the recorded customer response 

3 comprises: 

4 a. means for identifying the resulting state of the customer; 

5 b. means for updating the values of the states of the customer; and 

6 c. means for updating the optimal policy. 

1 21. A computer program product for dynamically developing a marketing strategy to address 

2 at least one specified merchant objective, the objective corresponding to a specified time 

3 period and a specified budget, the strategy being implemented across at least one 

4 marketing channel, the strategy including at least one initiative, the computer program 

5 product comprising: 

6 a. program instruction means for generating a plurality of marketing strategies; 

7 b. program instruction means for determining an optimal marketing strategy based on 

8 state of a customer and constraints corresponding to marketing channels; 

9 c. program instruction means for deploying the determined optimal marketing 

10 strategy; 

11 d. program instruction means for recording customer response to the deployed 

12 marketing strategy; and 

13 e. program instruction means for updating information corresponding to the state of a 

14 customer based on the recorded customer response. 

1 22. The computer program product as recited in claim 21 wherein the program instruction 

2 means for generating marketing strategies comprises: 

3 a. program instruction means for selecting at least one initiative that enables the 

4 addressing of the specified objective; 

5 b. program instruction means for determining the sequences in which the selected 

6 initiatives can be deployed, if more than one initiative is selected; and 

7 c. program instruction means for combining the initiatives in the determined 

8 sequences to obtain the plurality of marketing strategies. 
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1 23. The computer program product as recited in claim 21 wherein the program instruction 

2 means for determining an optimal marketing strategy comprises: 

3 a. program instruction means for determining all possible states of customers; 

4 b. program instruction means for determining an optimal policy for each state based 

5 on past data; 

6 c. program instruction means for identifying the state of a customer, the customer 

7 visiting the merchant or the customer being selected from a database of customers; 

8 d. program instruction means for identifying the optimal policy for the identified 

9 customer state; 

10 e. program instruction means for modeling customer's preferences for marketing 

1 1 channels, cost and effectiveness of different marketing channels, and the specified 

12 budget as effective constraints; 

13 f . program instruction means for determining an optimal feasible policy based on 

14 effective constraints corresponding to marketing channels; and 

15 g. program instruction means for determining the optimal marketing strategy from 

16 the optimal feasible policy. 

1 24. The computer program product as recited in claim 23 wherein the program instruction 

2 means for determining the optimal policy for each state based on past data comprises: 

3 a. program instruction means for identifying a deterministic policy; 

4 b. program instruction means for initializing the value of all possible states for the 

5 policy; 

6 c. program instruction means for computing the value of a state for the policy; 

7 d. program instruction means for constructing a new improved policy; 

8 e. program instruction means for iteratively executing means for computing and 

9 means for constructing; and 

10 f. program instruction means for selecting the policy with maximum value for the 

1 1 state as the optimal policy for the given state. 

1 25. The computer program product as recited in claim 24 wherein the program instruction 

2 means for constructing a new improved policy comprises program instruction means for 
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3 selecting the marketing strategy that maximizes the value for the state over all marketing 

4 strategies for a given state. 

1 26. The computer program product as recited in claim 21 wherein the program instruction 

2 means for updating information corresponding to the state of a customer based on the 

3 recorded customer response comprises: 

4 a. program instruction means for identifying the resulting state of the customer; 

5 b. program instruction means for updating the values of the states of the customer; 

6 and 

7 c. program instruction means for updating the optimal policy. 

1 27. A system suitable for developing an optimal marketing strategy, the system comprising: 

2 a. a database storing information regarding initiatives that can be offered to 

3 customers, marketing channels available for executing the initiatives, cost and 

4 effectiveness of the marketing channels, and states of customers; 

5 b. means for enabling the merchant to specify at least one objective for a specified 

6 time period; 

7 c. means for generating a plurality of marketing strategies based on the objective 

8 specified by the merchant, the marketing strategies being a combination of 

9 initiatives; and 

10 d. means for determining the optimal marketing strategy and the corresponding at 

1 1 least one marketing channel based on state of the customer and cost and 

12 effectiveness of marketing channels. 

1 28. A method for dynamically developing a marketing strategy to address at least one 

2 specified merchant objective, the objective corresponding to a specified time period and a 

3 specified budget, the strategy being implemented across at least one marketing channel, 

4 the strategy including at least one initiative, the method comprising the steps of: 

5 a. generating a plurality of marketing strategies; 

6 b. determining all possible states of customers; 

7 c. determining an optimal policy for each state based on past data; 

8 d. identifying the state of a customer, the customer visiting the merchant or the 

9 customer being selected from a database of customers; 
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identifying the resulting state of the customer; 
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updating the values of the states of the customer; 


20 


m. 


updating the optimal policy; and 
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repeating steps c to m for the specified time period. 
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